Skip to content
This repository has been archived by the owner on Jan 25, 2023. It is now read-only.

Make opentype tables compatible with Universal Shaping Engine for scripts routed to USE #576

Closed
jungshik opened this issue Nov 26, 2015 · 27 comments

Comments

@jungshik
Copy link
Contributor

Spun off from #543, notofonts/balinese#22, notofonts/lepcha#3, and notofonts/tifinagh#12.

We need to make Phase 3 fonts for scripts that are routed to USE be USE-compatible.

https://www.microsoft.com/typography/OpenTypeDev/USE/intro.htm

/cc @roozbehp @tiroj @marekjez86 @behdad @TrueTyper @waksmonskiMT @BrianKraimerMT

@jungshik jungshik added this to the Phase III milestone Nov 26, 2015
@jungshik jungshik changed the title Make opentype tables compatible with USE for scripts routed to USE Make opentype tables compatible with Universal Shaping Engine for scripts routed to USE Dec 9, 2015
@jungshik
Copy link
Contributor Author

jungshik commented Dec 9, 2015

Scripts that go to USE (Universal Shaping Engine) are listed below. (taken from @behdad's comment : #543 (comment) )

Balinese
Batak
Brahmi
Buginese
Buhid
Hanunoo
Kaithi
Kharoshthi
Lepcha
Limbu
MeeteiMayek
Rejang
Saurashtra
Sundanese
SylotiNagri
Tagalog
Tagbanwa
TaiLe
TaiViet

/cc @jamesgk

@jungshik jungshik assigned waksmonskiMT and unassigned kmansourMT Dec 9, 2015
@jungshik
Copy link
Contributor Author

jungshik commented Dec 9, 2015

@waksmonskiMT

@jungshik
Copy link
Contributor Author

jungshik commented Dec 9, 2015

/cc @kmansourMT

@xiangyexiao
Copy link
Contributor

We expect to get the fix by end of April. The early the better. Is it feasible?

@kmansourMT
Copy link

Xiangye, please clarify:

What fix will be ready by end of April?

To whom is the following question directed: Is it feasible?

From: Xiangye Xiao <notifications@github.commailto:notifications@github.com>
Reply-To: googlei18n/noto-fonts <reply@reply.github.commailto:reply@reply.github.com>
Date: Thursday, 25 February 2016 at 15:28
To: googlei18n/noto-fonts <noto-fonts@noreply.github.commailto:noto-fonts@noreply.github.com>
Cc: Kamal Mansour <kamal.mansour@monotype.commailto:kamal.mansour@monotype.com>
Subject: Re: [noto-fonts] Make opentype tables compatible with Universal Shaping Engine for scripts routed to USE (#576)

We expect to get the fix by end of April. The early the better. Is it feasible?


Reply to this email directly or view it on GitHubhttps://github.com//issues/576#issuecomment-189033563.

@xiangyexiao
Copy link
Contributor

sorry for the confusion. We (Jungshik) already fixed the bugs for Phase II fonts. The timeline in my last comment is for ourselves to update Android with the new fonts.

@xiangyexiao
Copy link
Contributor

still assign to Sue since this bug is for Phase III.

@jungshik
Copy link
Contributor Author

jungshik commented Mar 2, 2016

It cannot be priority critical. This should happen as Phase III fonts are being developed.

@roozbehp
Copy link
Contributor

roozbehp commented Mar 2, 2016

You're correct Jungshik. I failed to notice that the bug is about Phase 3 fonts.

@nizarsq
Copy link
Collaborator

nizarsq commented Aug 14, 2020

@marekjez86 what is the status? suggesting on how I should create test case?

@behdad

This comment has been minimized.

@punchcutter
Copy link

Most scripts were updated for phase III when USE was actually getting implemented. I only see a few now that don't have the script specific tag and only use DFLT. For those I only know the reason for Zanabazar Square as stated below. Elymaic I don't know. Tifinagh is a question for JamraPatel since they did it. Tai Tham has been in discussion for a long time about how to handle it in USE so it's not really working 100% anyway. So I think this issue should definitely be closed and new issues can be opened if there's an issue with any particular script.

ADLaM ✅
Ahom ✅
Bhaiksuki ✅
Balinese ✅
Batak ✅
Brahmi ✅
Buginese ✅
Buhid ✅
Chakma ✅
Cham ✅
Chorasmian - no font
Dives Akuru - no font
Dogra ✅
Duployan - no features, no script tags
Egyptian Hieroglyphs - no features, no script tags
Elymaic ❌
Grantha ✅
Gunjala Gondi ✅
Hanifi Rohingya ✅
Hanunoo ✅
Javanese ✅
Kaithi ✅
Kayah Li ✅
Kharoshthi ✅
Khitan Small Script - no font
Khojki ✅
Khudawadi ✅
Lepcha ✅
Limbu ✅
Mahajani ✅
Makasar - no font
Mandaic ✅
Manichaean ✅
Marchen ✅
Masaram Gondi ✅
Medefaidrin - no font
Meitei Mayek ✅
Miao ✅
Modi ✅
Mongolian ✅
Multani - no features, no script tags
Nandinagari - no font
Newa ✅
N’Ko ✅
Nyiakeng Puachue Hmong ✅
Old Sogdian ✅
Pahawh Hmong ✅
Phags-pa ✅
Psalter Pahlavi ✅
Rejang ✅
Saurashtra ✅
Sharada ✅
Siddham ✅
Sinhala ✅
Sogdian ✅
Soyombo ✅
Sundanese ✅
Syloti Nagri ✅
Tagalog ✅
Tagbanwa - no features, no script tags
Tai Le ✅
*Tai Tham ❌
Tai Viet ✅
Takri ✅
Tibetan ✅
Tifinagh ❌
Tirhuta ✅
Wancho ✅
Yezidi - no font
Zanabazar Square ❌ because of problems with categories and USE shaping implementation. @dscorbett does the same as explained here: https://github.com/dscorbett/zanashinar#script-and-language

@davelab6
Copy link
Member

I propose we keep this issue open until all are green checks or there is a separate issue that describes what the script specific situation is and how to fix it

@BrianKraimerMT
Copy link

BrianKraimerMT commented Aug 17, 2020 via email

@davelab6
Copy link
Member

@behdad Noto is actively maintained, so saying no one has bandwidth isn't really fair. Perhaps that was true in the intervening past few years.

As to the accusation that Monotype didn't deliver, I have to say that their invoices were all approved and they remain in good standing with Google. Zachary has given the receipts, so I'd like to kindly suggest that you apologise.

Generally, type designer firms and folks are service providers, so if the client hasn't specified up front what their requirements are, then a positive atmosphere is required to collaborate on negotiating what is and what is not within the scope of an agreed service provision engagement. When working at the forefront of technology, it can be impossible to specify requirements up front.

USE was very fresh at that time, so I think the situation today is fine.

@kmansourMT
Copy link

kmansourMT commented Aug 17, 2020 via email

@tiroj
Copy link

tiroj commented Aug 17, 2020

As for Elymaic, it is an off-shoot of Aramaic script. Make it conform to USE make as little sense as doing so for Hebrew script. Tifinagh has no features that require USE since it is a simple script (comparable to Latin) that requires a fair number of combining marks.

Windows 10 passes Tifinagh to USE. Not sure what it does with Elymaic, but it would make some sense to pass it to the Hebrew shaping engine (as is done with Thaana on the grounds that it happens to use the same layout model despite being completely unrelated to Hebrew).

While USE provides some specific modelling of shaping for complex scripts, I think it is a mistake to assume that non-complex scripts should not be passed to USE in the absence of another explicitly identified shaping engine route. In addition to its complex script shaping models, USE provides access to OTL features generally—including standard typographic and discretionary features — in a way that is more consistent than in other shaping engines. While it is obviously up to platform developers to determine how to route specific runs after itemisation, my own inclination would be to pass any script tag that isn't already linked to a specific shaping engine via USE, as this provides the most complete and flexible support for OTL features that may occur in any font. It is, after all, called the Universal Shaping Engine.

@NorbertLindenberg
Copy link

Elymaic is listed in the documentation of the Universal Shaping Engine:
https://docs.microsoft.com/en-us/typography/script-development/use
The USE covers several other right-to-left scripts.

@kmansourMT
Copy link

kmansourMT commented Aug 17, 2020 via email

@NorbertLindenberg
Copy link

First, ccmp and liga are not script-independent: They are not documented for all shaping engines, and while HarfBuzz applies them across scripts (at least for horizontal text), Windows doesn’t.

Second, possible issues when moving from the simple shaping engine to USE:

  • The USE applies a cluster model based on Unicode character data and numerous overrides, and inserts dotted circles where incoming text doesn’t match that model. The simple shaping engine doesn’t do that.

  • The USE applies features in the basic cluster formation phase on a per-cluster basis; features in later phases on a per-run basis. ccmp is applied in the basic cluster formation phase, so lookups can’t see beyond cluster boundaries. For the simple shaping engine, this is not specified, but implementations seem to apply lookups to the entire run. As a result, some lookups that work fine in the simple shaping engine may not work in the USE.

@dscorbett
Copy link

There is no need to keep this issue open. The fonts for Elymaic, Tifinagh, and Zanabazar Square work fine with the default shaper. USE supports those scripts, but that doesn’t mean font designers have to use it; it’s just an option if they want to. Switching now has no benefit and could introduce subtle bugs. Noto Sans Tai Tham is broken, but moving to USE would probably make fixing it harder, since USE doesn’t really support Tai Tham.

@davelab6
Copy link
Member

There is no need to keep this issue open. The fonts for Elymaic, Tifinagh, and Zanabazar Square work fine with the default shaper. USE supports those scripts, but that doesn’t mean font designers have to use it; it’s just an option if they want to. Switching now has no benefit and could introduce subtle bugs.

Alright, sounds good. It seems like this ought to go into future detailed Noto docs (cc @twardoch)

Noto Sans Tai Tham is broken, but moving to USE would probably make fixing it harder, since USE doesn’t really support Tai Tham.

What's the status of fixing USE to properly support Tai Tham? Currently no momentum and at a dead stop? Going to be fixed this year? :)

@roozbehp
Copy link
Contributor

What's the status of fixing USE to properly support Tai Tham? Currently no momentum and at a dead stop? Going to be fixed this year? :)

Pending action by Unicode. We last met specifically about it in late 2018 to discuss. Report here:
https://www.unicode.org/L2/L2018/18332-tai-tham-ad-hoc-report.pdf

One of our key contributors, Martin Hosken, who is our key contact on the ground, has been blocked by his work on Western Cham. We are mostly waiting on him, as well for others to finish their action items.

@davelab6
Copy link
Member

Thanks roozbeh!

@davelab6
Copy link
Member

@nizarsq asked

suggesting on how I should create test case?

@dscorbett and all, do you have ideas for such test cases? :)

@dscorbett
Copy link

This issue is not a bug report: it’s a reminder to check that all Noto fonts that use USE do so properly. As such, it won’t have a test case like other issues, which I assume is what @nizarsq was asking about. Test cases can be created for specific incompatibilities if and when they are discovered.

@marekjez86
Copy link
Contributor

@punchcutter @NorbertLindenberg @davelab6 @roozbehp @dscorbett @tiroj @kmansourMT @nizarsq : I wanted to thank you all for the contribution to this issue. I'll use the table created by @punchcutter to either create individual script issues or as a reminder to check a new script (or set requirements). Closing this issue.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests